3d Sound Can Have a Negative Impact on the Perception of Visual Content in Audiovisual Reproductions

There is reason to believe that sound interacts with visual attention mechanisms. Practical implications of that interaction have never been analyzed in the context of spatial sound design for audiovisual reproduction. The study reported here aimed to test if sound spatialization could affect eye movements and the processing of visual events in audiovisual scenes. We presented participants with audiovisual scenes of a metro station. The sound was either mono, stereo, or 3D. Participants wore eye tracking glasses during the experiment and their task was to count how many people entered the metro. In the divided attention task, participants had to count people entering 3 doors of the metro. In the selective attention task, participants had to count how many people entered the middle door alone. It was found that sound spatialization did not affect the divided attention task. But in the selective attention task participants counted less visual events with 3D sound. In that condition, the number of eye fixations and time spent in the visual area of interest were smaller. It is hypothesized that, in the case of divided attention, the attention is already disengaged and fluctuating, which could explain why sound did not have any additional effect. In the selective attention task, participants must remain concentrated in only one visual area and competing well-spatialized sounds in peripheral areas might have a negative impact. These results should be taken into consideration when designing sound spatialization algorithms and soundtracks.